1. Field of the Invention
The present invention generally relates to the field of computer-based testing, and in particular, the present invention relates to amalgamating an exam resource file to combine first defined and later defined test content and specification of a computer-based test to reduce the amount of authored XXL and to reduce the size of the size of the exam resource file during delivery of the computer-based test to an examinee.
2. Background of the Related Art
For many years, standardized testing has been a common method of assessing examinees as regards educational placement, skill evaluation, etc. Due to the prevalence and mass distribution of standardized tests, computer based testing has emerged as a superior method for providing standardized tests, guaranteeing accurate scoring, and ensuring prompt return of test results to examinees.
Tests are developed based on the requirements and particulars of test developers. Typically, test developers employ psychometricians or statisticians and psychologists to determine the specific requirements specific to human assessment. These experts often have their own, unique ideas regarding how a test should be presented and regarding the necessary contents of that test, including the visual format of the test as well as the data content of the test. Therefore, a particular computer based test has to be customized to fulfill the client's requirements.
FIG. 1 illustrates a prior art process for computerized test customization, denoted generally by reference numeral 10. First, a client details the desired test requirements and specifications, step 12. The computerized test publisher then creates the tools that allow the test publisher to author the items, presentations, etc., required to fulfill the requirements, step 14. The test publisher then writes an item viewer, which allows the test publisher to preview what is being authored, step 16.
An item presenter is then written to present the new item, for example, to the test driver, step 18. Presenting the new item to the test driver requires a modification of the test driver's executable code. The test driver must be modified so that it is aware of the new item and can communicate with the new item presenter, step 20. The test packager must then also be modified, step 22. The test packager, which may also be a compiler, takes what the test publisher has created and writes the result as new object codes for the new syntax. Subsequently, the scoring engine must also be modified to be able to score the new item type, step 24. Finally, the results processor must be modified to be able to accept the new results from the new item, step 26. This process requires no less than seven software creations or modifications to existing software.
During authoring of the test, the test author might desire to define some portion of the test specification for re-use. Alternatively the test author might redefine or override certain portions of the test specification that were defined early in the test specification. Previous test definition languages attempted to solve this problem by repeating selected pieces of the test specification at a higher level of the delivery hierarchy. However, it has been determined that this solution does not cover the content of the test specification and assumes that the pattern of repetition is based on the unit of delivery. For example, test specification based on the order of delivery and the hierarchy may be: a two-section exam of math followed by English. All math questions have a minimum response value of 2 and all English questions minimum response value of 3. An example that will not support repetition of test definition is a two-section exam of math and English. Where half the math questions have a minimum response of 2 and the other half of the math questions have a minimum response of 3. In this situation, the hierarchy does not match the pattern of repetition.
Furthermore, it has been determined that this solution requires specific units of the test to have detailed knowledge of shared specification and to communicate the shared specification to other units of the test.
U.S. Pat. No. 5,827,070 (Kershaw et al.) and U.S. Pat. No. 5,565,316 (Kershaw et al.) are incorporated herein by reference. The '070 and '316 patents, which have similar specifications, disclose a computer-based testing system comprising a test development system and a test delivery system. The test development system comprises a test document creation system for specifying the test contents, an item preparation system for computerizing each of the items in the test, a test preparation system for preparing a computerized test, and a test packaging system for combining all of the items and test components into a computerized test package. The computerized test package is then delivered to authorized examinees on a workstation by the test delivery system.
FIG. 2 illustrates the relationship among session scripts 30, test scripts 32, and units. A script consists of a series of files and further specifies the option settings and configuration data, which the Test Delivery Application (TDA) needs for operation. During test preparation, scripts are prepared and combined with the items prepared during item preparation. Scripts control the sequence of events during a testing session. Two types of scripts are preferably used, for example: the session script 30 and one or more test scripts 32. The session script 30 controls the order in which units within the testing session are presented. Units provide specific services to the examinee, such as delivering a test or presenting a score report. Just as the session script controls the session, the test, script controls what is presented to the examinee during the testing unit. Each testing unit may include one or more delivery units, which are separately timed and scored subdivisions of a test. The system can dynamically select, or spiral, scripts and other test components so that examinees are given what appear to be different tests. FIG. 24 shows the relationship among session scripts 30, test scripts 32, and units.
The session script is the second-level component of the testing package. It performs two primary functions: First, it specifies the Session Control Information, which defines the default options that are in effect for the entire examinee testing session. Second, it controls the order in which units within the testing session are presented and the options used to present them. The units that can be presented within a session script are: General information screen units, Tutorial units, Break units, Data collection units, Scoring and Reporting units, and Testing units.
The session control information contains the default options in effect for the entire session. Control information can be provided at multiple levels within the testing session. Thus, the control information provided at the session level can be overridden by information that occurs later in the session. The information provided at the session level would generally include the following: Name—the session script name to be used by administrators in selecting a specific session script from Administrative Application menus; Input device—the input device to be used during the session (e.g., mouse or keyboard); Color—the colors to be used during the session; Messages—program-specific messages to override default messages during the session; Demo Script—indicates whether the script presents a demonstration or operational test; Research Indicator—indicates whether the script presents a research pilot test; Special Timing—indicates whether the script is standard or specially timed version.
The testing unit presents a test, based on the contents of a test script that may have been selected at runtime. The following units can be included within a testing unit: general information screen unit; tutorial unit; break unit; delivery unit, which delivers items to the examinee. This permits testing programs to interleave general information screens, tutorials, and breaks with sections of a test. The testing unit contains the following information: script selection mode indicates whether dynamic runtime selection is to be used to select the test script; reference to a test script which controls the sequence of events and options used during the testing unit. If dynamic runtime selection is to be used, the reference is to a set of test scripts. Like the session script, the test script performs two primary functions. First, it specifies the test and delivery unit control information. Test control information defines the options that are in effect for the testing unit. Delivery unit control information defines the options that are in effect for a particular delivery unit within a testing unit. It controls the order in which units are presented within the testing unit and the options used to present them. The rules for presentation of units are the same as those for the session script, except that an additional unit, the delivery unit, can be included within a test script.
U.S. Pat. No. 5,513,994 (Kershaw et al.), which is incorporated herein by reference, discloses a centralized administrative system and method of administering standardized test to a plurality of examinees. The administrative system is implemented on a central administration workstation and at least one test workstation located in different rooms at a test center. The administrative system software, which provides substantially administrative functions, is executed from the central administration workstation. The administrative system software, which provides function carried out in connection with a test session, is executed from the testing workstations.
None of the Kershaw et al. patents appear to make any mention of a test definition language that does not required the overriding language to be repeated at higher levels of the delivery hierarchy. What is required is a computer based testing system that supports a non-deterministic test definition language that, for example, allows subsequent specifications to be added to the test definition language without requiring repetitive definitions and that enables a reduction in size of exam source files, and an exam resource file, which contains the test definition language during delivery of the test. Other features and advantages in addition to the above, or in the alternative to the above, are described in the Summary of the Invention and the Detailed Descriptions provided below.